/home/tony/Work/glockenspiel/s3prl/s3prl/upstream/byol_s/byol_a/common.py:20: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
  torchaudio.set_audio_backend("sox_io")
2023-10-05 18:14:26 | WARNING | root | Pytorch pre-release version 2.1.0.dev20230831+cu118 - assuming intent to test it
/home/tony/Work/glockenspiel/s3prl/s3prl/run_downstream.py:157: UserWarning: torchaudio._backend.set_audio_backend has been deprecated. With dispatcher enabled, this function is no-op. You can remove the function call.
  torchaudio.set_audio_backend('sox_io')
[Featurizer] - Take a list of 13 features and weighted sum them.
[Featurizer] - The selected feature hidden_states's downsample rate is 960
[Runner] - Start a new experiment
mert_test_8x_400k {'model_config': None, 'refresh': False}
Loading model from Local mert_test_8x_400k
load local model succeed
load local feat extractor
Wav2Vec2FeatureExtractor {
  "do_normalize": false,
  "feature_extractor_type": "Wav2Vec2FeatureExtractor",
  "feature_size": 1,
  "padding_side": "right",
  "padding_value": 0,
  "return_attention_mask": true,
  "sampling_rate": 24000
}


overall:   0%|          | 0/4000 [00:00<?, ?it/s]

train:   0%|          | 0/37 [00:00<?, ?it/s]
overall:   0%|          | 1/4000 [00:02<2:17:54,  2.07s/it]

train:   3%|▎         | 1/37 [00:02<01:14,  2.07s/it]
overall:   0%|          | 2/4000 [00:02<1:22:35,  1.24s/it]

train:   5%|▌         | 2/37 [00:02<00:43,  1.24s/it]
overall:   0%|          | 3/4000 [00:03<1:04:31,  1.03it/s]

train:   8%|▊         | 3/37 [00:03<00:32,  1.03it/s]
overall:   0%|          | 4/4000 [00:04<55:51,  1.19it/s]  

train:  11%|█         | 4/37 [00:04<00:27,  1.19it/s]Process Process-12:
Process Process-9:
Process Process-7:
Process Process-4:
Process Process-6:
Process Process-1:
Process Process-2:
Process Process-8:
Process Process-10:
Process Process-11:
Process Process-5:
Process Process-3:
Exception ignored in: <function _MultiProcessingDataLoaderIter.__del__ at 0x7fd92bdf0160>
Traceback (most recent call last):
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1478, in __del__
    self._shutdown_workers()
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/torch/utils/data/dataloader.py", line 1442, in _shutdown_workers
    w.join(timeout=_utils.MP_STATUS_CHECK_INTERVAL)
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/multiprocessing/process.py", line 149, in join
    res = self._popen.wait(timeout)
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/multiprocessing/popen_fork.py", line 40, in wait
    if not wait([self.sentinel], timeout):
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/multiprocessing/connection.py", line 931, in wait
    ready = selector.select(timeout)
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/selectors.py", line 416, in select
    fd_event_list = self._selector.poll(timeout)
KeyboardInterrupt: 

train:  11%|█         | 4/37 [00:19<02:36,  4.76s/it]
Traceback (most recent call last):
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/transformers/feature_extraction_utils.py", line 175, in convert_to_tensors
    tensor = as_tensor(value)
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/transformers/feature_extraction_utils.py", line 167, in as_tensor
    return np.asarray(value, dtype=dtype)
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tony/Work/glockenspiel/s3prl/s3prl/run_downstream.py", line 219, in <module>
    main()
  File "/home/tony/Work/glockenspiel/s3prl/s3prl/run_downstream.py", line 215, in main
    eval(f'runner.{args.mode}')()
  File "/home/tony/Work/glockenspiel/s3prl/s3prl/downstream/runner.py", line 292, in train
    features = self.upstream.model(wavs)
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/tony/Work/glockenspiel/s3prl/s3prl/upstream/mert/expert.py", line 131, in forward
    input_values = self.extracter(
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/transformers/models/wav2vec2/feature_extraction_wav2vec2.py", line 199, in __call__
    padded_inputs = self.pad(
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/transformers/feature_extraction_sequence_utils.py", line 224, in pad
    return BatchFeature(batch_outputs, tensor_type=return_tensors)
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/transformers/feature_extraction_utils.py", line 78, in __init__
    self.convert_to_tensors(tensor_type=tensor_type)
  File "/home/tony/anaconda3/envs/suno_env/lib/python3.10/site-packages/transformers/feature_extraction_utils.py", line 181, in convert_to_tensors
    raise ValueError(
ValueError: Unable to create tensor, you should probably activate padding with 'padding=True' to have batched tensors with the same length.

overall:   0%|          | 4/4000 [00:19<5:20:59,  4.82s/it]